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Overview of the NAEP 2005 Mathematics Assessment 



Who was assessed? 

• The NAEP 2005 assessment was administered to a stratified random sample of 
fourth", eighth", and twelfth"graders at the national level and to a stratified random 
sample of fourth" and eighth"graders at the state level. 

• Both public and nonpublic school students were assessed at the national level. 

• At the state or jurisdiction level, only the results for public school students are 
reported. 

• Fifty"two jurisdictions participated, including the 50 states, the District of Columbia, 
and the Department of Defense Schools (Domestic and Overseas). 

• National (public and nonpublic) and state (public only) samples include the following: 

■ Grade 4 

• Approximately 172,000 students 

• Approximately 9,500 schools 

■ Grade 8 

• Approximately 161,600 students 

• Approximately 7,200 schools 

• Colorado sample number assessed: 

■ Grade 4 

• Approximately 2,800 students 

• Approximately 160 schools 

■ Grade 8 

• Approximately 2,500 students 

• Approximately 140 schools 

What is assessed? 

• The NAEP Frameworks specify what is assessed and how it is to be assessed. 

• The Mathematics Framework for NAEP was revised in 1996 and again in 2005. The 
new framework reflects current curricular emphases and objectives, while 
continuing a connection to previous frameworks. The connection allows the trend 
line at grades 4 and 8 that started with the 1990 assessment to be maintained. 

• The Mathematics Framework for NAEP describes the content and format of the 
2005 assessments (http://www.nagb.org/pubs/m_framework_05/toc.html). Revisions 
to the framework maintain the short"term trend, and results are comparable across 
all assessment years. 

• The NAEP Mathematics Framework describes content in five strands and the 
percentage of questions that should be devoted to each 



Target percentage distribution of questions 


Content Strand 


Grade 4 


Grade 8 


Number properties and operations 


40 


20 


Measurement 


20 


15 


Geometry 


15 


20 


Data analysis and probability 


10 


15 


Algebra 


15 


30 
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• The Mathematics Framework for NAEP 2005 also describes three levels of 
mathematical complexity that include aspects of knowing and doing mathematics. 

• Low Complexity 

■ Relies heavily on the recall and recognition of previously learned concepts and 
principles (recall, recognize, compute, perform) 

• Moderate Complexity 

■ Involve more flexibility of thinking and choice among alternatives; require a 
response that goes beyond the habitual. 

• High Complexity 

■ Require student to think in abstract and sophisticated ways; involves 
planning, analysis, judgment, and creative thought. 

• Approximately half of the score on the assessment is based on items of moderate 
complexity, with the remainder of the score based equally on items of low and high 
complexity. 

How is mathematics assessed? 

• Mathematics is assessed using three types of questions^ 

■ Multiple-choice questions — 

• make up 50 percent of the assessment, and 

• four choices are presented. 

■ Short, constructed-response questions — 

• “...require students to give either a numerical result or the correct name or 
classification for a group of mathematical objects, draw an example of a 
given concept, or perhaps write a brief explanation for a given result.” 

■ Extended, constructed-response questions — 

• demand more than a numerical or short verbal response; and 

• require students to carefully consider a problem, plan an approach, solve 
the problem, and interpret their solution in terms of the original problem. 

• In 2005, the distribution of types of questions across all blocks was as follows^ 



Type of Question 


Grade 4 


Grade 8 


Multiple-choice 


111 


122 


Short, constructed response 


52 


49 


Extended, constructed response 


7 


7 


Total 


170 


178 



• Calculators were provided for about one-third of the assessment (one-third of the 
blocks). 

■ Not all items in a calculator available assessment block require the use of a 
calculator. 

■ NAEP provides the calculators^ 

• Grade 4 — four-function calculator 

• Grade 8 — scientific calculator 
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■ Items in non-calculator blocks require students to demonstrate computation 
or estimation skills without a calculator. 

• Some items use manipulatives, e.g., rulers, protractors, spinners, and geometric 
shapes. The manipulatives are provided by NAEP. 

How are constructed-response questions scored? 

• Unique scoring guides are developed for each constructed-response question. 

• Scoring guides describe the specific criteria for assigning a score level for student 
responses. 

• Extended, constructed-response questions had four- and five-level scoring guides. 

• Many short, constructed-response questions had three-level guides that allowed for 
partial credit, while others were rated as either acceptable or unacceptable. 

Scoring process : 

■ Expert scorers are extensively trained to apply the scoring criteria 
consistently and fairly. 

■ Scoring is monitored to ensure the scoring standards are being adhered to 
reliably. 

■ Monitoring measures the consistency of scoring to the same items 
administered in different assessments — therefore, ensuring consistency of the 
application of scoring standards across assessment years. 

• Over 4,435,831 mathematics constructed responses were scored for the 2005 
assessment. 

How is the NAEP mathematics assessment administered? 

• Each student took two, 25-minute sets of mathematics questions (also referred to as 
blocks), one set of general background questions, and one set of background 
questions related to mathematics. 

• In order to provide a comprehensive assessment of mathematics and to minimize the 
burden on any individual student, NAEP uses matrix sampling. Each student takes 
a subset of the total set of questions, i.e., two blocks out of ten total blocks per grade 
level. 

• Because each block is spiraled with other blocks and is administered to a 
representative sample of students, the results can be combined to produce average 
group and subgroup results based on the entire assessment. 

How long does the NAEP assessment take? 

• No more than about 1 hour per student to actually take the assessment — about 50 
minutes on mathematics questions, and a few more minutes on background 
questions. 



^ Sources- 

Appendix A. Overview of Procedures Used for the NAEP 2005 Mathematics Assessment, State Report Generator. 
NAGB (2004) Mathematics Framework for the 2005 National Assessment of Educational Progress. Washington, DC- 
Author 
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Variables/Factors Available in NAEP Reporting 



NAEP started in 1964 through a grant from the Carnegie Corporation; that set up the 
Exploratory Committee for the Assessment of Progress in Education. The first national 
assessments were conducted in 1969 under the U.S. Department of Education. 
Voluntary assessments for states began in 1990. In 2003, NCLB mandated NAEP 
grades 4 and 8 Reading and Mathematics Assessments. 

NAEP is not designed to report individual students, school or district data due to a 
number of reasons including: sample size, students do not take the entire test and the 
data are only estimates of selected sample population subgroups. Therefore only state 
and national results are available. 

NAEP/Westat, the contractor for NAEP data submittal uses a number of student 
variables to report on the subject-matter achievement of selected sample population 
subgroups. These variables are: 

• Gender 

• National School Lunch Program status 

• Race/ethnicity 

• Limited English Proficiency status 

• Individualized Educational Plan or 504 status 

In addition, NAEP surveys the school administrator, teachers of assessed students and 
the participating students on a number of factors including but not limited to: 

• School Location and Demographics 

• Teacher Demographics 

• Teacher Preparation, Credentials, Experiences 

• Student Time Use Outside of School 

• Parental Education Level 

A number of these variables/factors are reported for Colorado grades 4 and 8 in this 
document and are certainly not inclusive of the vast quantity of data available from 
NAEP. For access to more data please visit the NAEP Data Explorer tool on-line at 
http://nces.ed.sfov/nationsreportcard/nde/ 
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Types of Scoring 



Two different ways NAEP reports scores is represented in this document^ Average Scale 
Score and Achievement Levels. Following is a brief description of each. 

Average Scale Score 

• Scale scores — indicate how much students know and can do. 

■ Range is 0-500 

■ The Average Scale Score reflects the subgroup as a whole and is not 
necessarily the Average Scale Score achieved by those students at or above 
Proficient. 



NAEP Mathematics Cut Scores 


_ ^th 


Grade 


Advanced (A) 


282 


- 500 


Proficient (P) 


249 


- 281 


Basic (B) 


214 


- 248 


Below Basic (BB) 


0 - 


213 



NAEP Mathematics Cut Scores 


- 8*1^ Grade 


Advanced (A) 


333 - 500 


Proficient (P) 


299 - 332 


Basic (B) 


262 - 298 


Below Basic (BB) 


0 - 261 



Achievement Levels 



Policy definitions of NAEP Achievement Levels- 

• Basic- This level denotes partial mastery of prerequisite knowledge and skills that 
are fundamental for proficient work at each grade. 

• Proficient'- This level represents solid academic performance for each grade assessed. 
Students reaching this level have demonstrated competency over challenging subject 
matter, including subject-matter knowledge, application of such knowledge to real- 
world situations, and analytical skills appropriate to the subject matter. 

• Advanced: This level signifies superior performance. 

For more details on the NAEP Achievement Level descriptions for mathematics — set 

separately by grade see the following indicated source. 



Source: National Assessment Governing Board. (2004) Mathematics Framework for the 2005 
National Assessment of Educational Progress. Washington, DC: Author 
http://www.nagh. org/puhs/m framework 05/toc.html 
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NAEP Grade 4 Math T rend: CO vs. Nation 




NAEP Grade 8 Math: CO vs. Nation 




NOTE: The NAEP Mathematics scale ranges from 0 to 500. Observed differences are 
not necessarily statistically significant. SOURCE: U.S. Department of Education, 
Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP), 1990, 1992, 1996, 2000, 2003 and 
2005 Mathematics Assessments. 
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stem and Leaf Table Comparing Colorado’s 
Grade 4 Percentage of Students At and Above Proficient 
NAEP 2005 Mathematics 



Previous Years 


% At or Above 
Proficient 


States and Other Jurisdictions 




49 


Massachusetts (48.8) 




48 






47 


Minnesota (47.3), New Hampshire (46.9), Kansas (46.8) 




46 






45 


New Jersey (45.4) 




44 


Vermont (43.5) 




43 


Wyoming (42.6), Ohio (42.5), Connecticut (42.5) 




42 


Washington (41.6), Pennsylvania (41.5) 




41 


South Dakota (40.6) 




40 


Idaho (40.4), North Dakota (40.4), Wisconsin (40.3), Texas 
(40), North Carolina (39.9) 


COLORADO 2005 


39 


Virginia (39.3), Maine (38.8) 




38 


Montana (38.3), Indiana (38.2), Maryland (38), Michigan (37.7) 




37 


Iowa (37.3), Oregon (37), Utah (36.8), Florida (36.6) 




36 


Delaware (36.1), Nebraska (36.1), New York (36.1), South 
Carolina (35.9) 


National 2005 


35 


Dept of Defense (34.8) 


COLORADO 2003 


34 






33 






32 


Illinois (31.6) 


National 2003 


31 


Missouri (31.1), Rhode Island (30.5) 




30 


Georgia (29.5) 




29 


Oklahoma (28.6) 




28 


California, Arizona (27.9), Tennessee (27.7) 




27 


Hawaii (26.7), 




26 


Nevada (26.1), Kentucky (26.1) 




25 


West Virginia (25.1) 




24 


Louisiana (23.9) 


National 2000 


23 






21 


Alabama (20.9) 




20 




National 1996 


19 


Mississippi (19.4), New Mexico (19) 


COLORADO 1992 


18 




National 1992 


17 






16 






15 






14 






13 




National 1990 


12 






11 






10 


District of Columbia (9.6) 


Higher than COLORA 


DO Not signil 

color; 


ficantly different from Lower than COLORADO 

VDO 
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stem and Leaf Table Comparing Colorado’s 
Grade 8 Percentage of Students At and Above Proficient 
NAEP 2005 Mathematics 



Previous Years 


% At or 
Above 
Proficient 


States and Other Jurisdictions 




44 






43 


Massachusetts (43.3), Minnesota (42.7) 




42 






41 






40 






39 






38 


Vermont 




37 


South Dakota (36.5) 




36 


Montana Washington, New Jersey (35.9), Wisconsin 
(35.8) 




35 


Nebraska (34.9), North Dakota (34.6), Connecticut (34.6), 
New Hampshire (34.6) 


COLORADO 2003 


34 


Kansas (34.2), Iowa (33.7), Oregon (33.7), 




33 


Virginia (33.4), Ohio (33.1), D of Defense (32.6) 


COLORADO 2005 


32 


North Carolina (31 .9) 




31 


Pennsylvania (30.9), New York (30.8), Texas (30.7) 




30 


Indiana (30.4), Idaho, South Carolina (29.9), Maine (29.9) 
Delaware (29.7), Maryland (29.6), Utah (29.5) 




29 


Michigan (29.3), Wyoming, Alaska (28.7), Illinois (28.6) 




28 




National 2003 


27 






26 


Missouri, Arizona (25.7), Florida (25.6) 


National COLORADO 
2000 1996 


25 






24 


Rhode Island 


National 1996 


23 


Georgia (23.2), Kentucky (22.5) 


COLORADO 1992 


22 


Arkansas, California (21.7) 




21 


Nevada (21 .4) Oklahoma (20.6), Tennessee (20.6) 


National 1992 


20 






19 






18 


Hawaii (18.2), West Virginia (17.9) 


COLORADO 1990 


17 






16 


Louisiana 




15 


Alabama 




14 


New Mexico, Mississippi (13.6) 




13 






12 






11 






10 






9 






8 






7 


District of Columbia 


Higher than COLORADO Not significantly different from Lower than COLORADO 

COLORADO 
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ALL STUDENTS - AVERAGE SCALE SCORE 



Math Grade 4 




I I HaCiDn Ipublic) 



DistricC af Columbia 

I I DsaDEA^ 



■'CJ Focal stafteii'jurisdiction (Colonado';i 

I I Highef average scale score Uhan Colorado (1 D jurisdictions) 

I I Nol signilicantty different from Colorado (nation and 22 jurisdiclions) 

Lower average scale score than Colorado (19 junsdictionsj 

^ Department of Defense Education Actiwitv schools (domestic and overseas). 

SOURCE: U.S. Department of Education. Instiluie of Education Sciences. National Cenier for Education Statistics, National Assessment 
of Educational Progress (NAEP). 2005 Malhematics Assessment. 



Math Grade 8 




KM Focal state^jurisdiction (Colorado) 

I I Higher average scale score Uhan Colorado (11 jurisdictions) 

I I Nol significantly different from Colorado (22 jurisdictions) 

Lower average scale score than Colorado (nalion and IS jurisdictions) 

1 Departmentof Defense Education Acliwfy schools (domestic and overseas). 

SOURCE: U.S. Department of Education. Institute of Education Sciences. National Center for Education Statistics, National Assessment 
of Educational Progress (ISIAEP). 2005 Mathematics Assessment. 
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NAEP Achievement by National School Lunch Eligibility 



60 % 

50 % 

40 % 

30 % 

20 % 

10% 

0% 

Eligible: Free/Reduced Not Eligible 



CO Math Grade 4 Free/ Reduced Lunch Status □ 2003 

2003 to 2005: Percent At or Above Proficient ■ 2005 




CO Math Grade 8 Free/ Reduced Lunch Status □ 2003 

2003 to 2005: Percent At or Above Proficient ■ 2005 



50 % 

40 % 

30 % 

20% 

10% 

0% 




Eligible: Free/Reduced Not Eligible 
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